In video surveillance applications, it is often desirable to retrieve or identify an object of interest, such as a vehicle, from images based on the color of the object. The problem is challenging because objects can have many poses with respect to the camera, and the color of the object as seen from the camera depends on the type of object, lighting conditions, pose, and camera characteristics. Existing color-based retrieval techniques are primarily from the field of content-based image retrieval (CBIR) and typically aim to search image databases for specific images that are similar to a given query image based on matching features derived from the image content. See, for example, M. V. Suhhamani and C.R. Venugopal, “Grouping and Indexing Color Features for Efficient Image Retrieval,” World Academy of Science, Engineering and Technology 27 (2007).
Color is often the most significant and distinguishing visual feature for retrieval. A number of techniques have been proposed or suggested for performing color classification of people and vehicles for video surveillance. See, for example, L. M. Brown, “Color Retrieval for Video Surveillance,” Advanced Video and Signal Based Surveillance (AVSS), Albuquerque, N. Mex., September 2009, incorporated by reference herein. Generally, objects are first segmented based on background subtraction and tracking. Objects are then classified as one of a predefined set of colors based on their histogram. In L. M. Brown, the histogram is based on an HSL (hue, saturation, and lightness) space and the primary color of the object is determined using a rule-based approach.
This type of color classification can cause a number of problems. The first issue is color constancy. Although people perceive an object to be the same color across a wide range of illumination conditions, the actual pixels of an object may have values that range across the color spectrum depending on the lighting conditions and relative pose. In addition, it may be difficult to accurately segment moving objects from the background. Shadows are often part of the object and errors exist in the segmentation due to the similarity of the object of interest, other objects in the scene and the background model. Further, complex objects may not be predominately one color. Certain aspects of objects are of interest to the human and these depend on the type of object and application.
It is often desirable to specify an object of interest for retrieval by a video surveillance system by identifying a similar object. For example, a user may identify a vehicle of interest by specifying the color, manufacturer and model of the desired vehicle. A need therefore exists for an example-based color retrieval system, where a selected example is employed to restrict alerts of candidate objects to “similar” objects. Among other benefits, an example-based color retrieval system does not require an object to be defined into a single color class. In addition, to the extent that the query and retrieved events occur under similar conditions, the example-based approach can reduce the issues that arise due to lighting variations.
A further need exists for an example-based retrieval system that can retrieve objects in real-time from a specific alarm-based set of events rather than from the set of all tracked objects stored in a database. In this manner, the errors based on segmentation and tracking are significantly reduced (since not all objects need to be tracked) while minimizing the variations due to pose.